Autonomous robotic system for placing and fastening paneling material for building construction operations

ABSTRACT

An autonomous construction robotic system is disclosed which includes a processing unit, a robotic arm, the robotic arm is adapted to be coupled to a central attachment arm and thereby position the central attachment arm according to a plurality of degrees of freedom, a panel handling and fastening system, including a panel handling assembly coupled to the central attachment arm and adapted to pick and place a construction panel onto a framed structure within a construction zone, and a vision system adapted to provide visual information to the processing unit associated with the framed structure, wherein the processing unit processes the visual information to automatically determine placement position of the construction panel on the framed structure.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present patent application is related to and claims the prioritybenefit of U.S. Provisional Patent Application Ser. No. 63/006,674 filedApr. 7, 2020, the contents of which are hereby incorporated by referencein its entirety into the present disclosure.

STATEMENT REGARDING GOVERNMENT FUNDING

This invention was made with government support under 182773311P awardedby National Science Foundation. The government has certain rights in theinvention.

TECHNICAL FIELD

The present disclosure generally relates to construction technology, andin particular, to an autonomous or a semi-autonomous robotic systemcapable of attaching paneling on the framed walls in a constructionsetting.

BACKGROUND

This section introduces aspects that may help facilitate a betterunderstanding of the disclosure. Accordingly, these statements are to beread in this light and are not to be understood as admissions about whatis or is not prior art.

For many decades now, construction in residential and commercialbuildings involve erecting framed walls; and once the framing operationis complete to hang paneling material on the interior surfaces as wellas exterior surfaces. These paneling materials include drywall as wellas other paneling materials such as foam. The paneling material aretypically in standard sizes and tend to be heavy. For example a typical⅝″ drywall is about 2.2 Lb/ft² making a sheet of 4 ft by 10 ftapproximately 88 Lbs. Such a weight is difficult to lift for a worker,in addition to the sheer size of such a panel, making handling a panelof this size challenging, particularly when installing such a panel on aceiling. A typical operation requires at least two individuals where oneor both place the panel at an appropriate place and quickly fastenusing, typically nails, until the panel is securely attached to theframed wall or ceiling. Thereafter, the worker(s) begin to fasten thedrywall panel with additional fasteners which are typically drywallscrews. The nails, however, have a tendency of popping out after aperiod of time making imperfections on the surface of the drywall. Inaddition, it is quite difficult to maintain a high level of consistencyin the way the screws are placed in the drywall panel, thereby makingthe finishing more difficult.

To alleviate some of the aforementioned challenges, a drywall lift isgenerally utilized. A drywall lift includes a platform upon which apanel of drywall is placed. Thereafter, a large wheel is rotated to liftthe panel to a proper height. While such a lift allows an easieroperation for the worker, the operation is even more time-consuming.Additionally, the drywall lift is bulky and difficult to manipulate.

while, in the last 30 years, worker productivity (measured in output perworker hour) in the manufacturing sector has increased by 120%,primarily due to advances in and adoption of automation technologies,worker productivity in single family home construction has beenstagnant, increasing by only 10% in the same period. This stagnation ofproductivity increase represents an unmet need.

Therefore, there is an unmet need for a novel approach and system thatcan place paneling materials on framed walls in a constructionenvironment that overcomes the aforementioned challenges.

SUMMARY

An autonomous construction robotic system is disclosed. The systeminclude a processing unit. The system further includes a robotic arm.The robotic arm is adapted to be coupled to a central attachment arm andthereby position the central attachment arm according to a plurality ofdegrees of freedom. The system also includes a panel handling andfastening system which includes a panel handling assembly coupled to thecentral attachment arm and adapted to pick and place a constructionpanel onto a framed structure within a construction zone. The systemfurther includes a vision system adapted to provide visual informationto the processing unit associated with the framed structure, wherein theprocessing unit processes the visual information to automaticallydetermine placement position of the construction panel on the framedstructure.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a schematic of a robotic arm system for placing and fasteningpaneling material on framed walls and ceilings, including a panelhandling and fastening system, according to the present disclosure.

FIG. 2 is a perspective schematic view of the panel handling andfastening system of FIG. 1.

FIGS. 3, 4, and 5 depict operation of the robotic arm system of FIG. 1.

FIG. 6 is a general schematic of a vision system with respect to thepanel handling and fastening system of FIG. 1.

FIG. 7 is a block diagram of a data processing system, according to thepresent disclosure.

FIG. 8 is an example of a computer system adapted to interface with thedata processing system of FIG. 7.

DETAILED DESCRIPTION

For the purposes of promoting an understanding of the principles of thepresent disclosure, reference will now be made to the embodimentsillustrated in the drawings, and specific language will be used todescribe the same. It will nevertheless be understood that no limitationof the scope of this disclosure is thereby intended.

In the present disclosure, the term “about” can allow for a degree ofvariability in a value or range, for example, within 10%, within 5%, orwithin 1% of a stated value or of a stated limit of a range.

In the present disclosure, the term “substantially” can allow for adegree of variability in a value or range, for example, within 90%,within 95%, or within 99% of a stated value or of a stated limit of arange.

A novel approach and system is disclosed herein that can place panelingmaterials on framed walls in a construction environment. Towards thisend, the novel system of the present disclosure includes an autonomousor a semi-autonomous robotic system which includes a computer visionsystem that can automatically i) locate a framed wall/ceiling within aconstruction zone, ii) establish a local coordinate system with respectto the framed wall/ceiling, iii) pick up a panel (e.g., a panel ofdrywall) from a stack of paneling positioned near the robotic system,iv) place the panel against the framed wall/ceiling, v) partially orcompletely fasten the panel against the framed wall/ceiling, vi) cutexcess portions of the panel while the panel is fastened on the framedwall/ceiling, vii) remove the excess portions of the panel and place ina location where debris from construction can be accumulated, and viii)complete fastening the panel if needed. The robotic system of thepresent disclosure is further and optionally configured to receive abuilding information modeling data file, known to a person havingordinary skill in the art, which includes framing information of wallswithin a structure; and use sensors mounted on the robotic system tomake measurements and generate a report of quality of construction forthe structure. Such sensors include a wide range of technology includingmicropower impulse radar, capacitive sensors that detect changes in walldensity, and force sensors that can be used to precisely locate thevertical members (called studs) within a framed wall/ceiling.

As mentioned, the novel system of the present disclosure is based on arobotic system. Typical industrial robotic arms are designed to operatein a tightly controlled environments, and typically utilize only alimited sensing capability which is tailored to a specific and narrowtask. However, an associated computer system which controls such arobot, conventionally has a limited ability to handle unexpectedconditions and will halt operation if too large a deviation fromexpected conditions are encountered.

Standard industrial robotic arm systems also generally lack the abilityto both manipulate and fasten a piece of material with a singlemanipulator arm, instead relying on multiple manipulators to accomplishsuch a task. This limits the portability and flexibility of suchsystems, and increases both cost and complexity of the electroniccontrols.

To alleviate the challenge of using a standard robotic system, thepresent disclosure provides a description of i) a robotic handling unit,ii) a robotic arm, and iii) an end effector that are all coupled to acentral attachment arm and are all further configured to work in concertwith one-another in order to achieve the aforementioned goals of thenovel robotic system of the present disclosure. The central attachmentarm is coupled to a commercially available robotic system capable ofgenerating motion in six degrees of freedom. Six degrees of freedomrepresent the minimum degrees of freedom needed to reach a volume ofspace from every available angle. Therefore, a system that can providesix degrees of freedom can preferentially reach each available angle toinstall a panel on a framed wall/ceiling.

Referring to FIG. 1, a schematic of the novel robotic arm system 100 forplacing and fastening paneling material on framed walls and ceilings isshown. The system includes two principal subsystems: (1) a robotic arm120 that can preferably generate six degrees of freedom to a connectingcomponent (although it is possible that a smaller number of degrees offreedom may also address the needs of certain constructions projects),and (2) a panel handling and fastening system 150 including a panelhandling assembly 160 and an end effector assembly 170, which are bothconfigured to be coupled to the robotic arm 120. The panel handling andfastening system 150 is further depicted in FIG. 2 which is aperspective schematic view of the panel handling and fastening system150 and which includes a central attachment arm 202, which is coupled toa plate 206. On the plate 206 there are one or more paneling pickuptools 204 _(i) (four are shown as 204 ₁, 204 ₂, 204 ₃, and 204 ₄,however, a smaller or larger number is also feasible). Example panelingpickup tools include suction cups coupled to a vacuum pump (not shown)drawing sufficient vacuum so as to overcome the weight of the panel.Example of weight of a 4 foot by 10 foot ⅝″ inch of drywall was providedin the background section. Therefore, to calculate the necessarypressure, the proportional weight of the panel can be divided by openingarea of the suction cup to establish the necessary pressure. Forexample, for an 88 Lbs drywall panel, being picked up by four suctioncups of for example 6″ each diameter, will require about 3.2 PSIG vacuumpressure: A 6″ diameter suction cup has an area of 28.27 in², thereforefour suction cups have an area of 113.1 in², thus to lift 88 pounds overan area of 113.1 in², we need a vacuum pressure of about 0.8 PSIGinnee—for vertical lifting, a factor of safety of four is recommended,that provides a required vacuum pressure of about 3.2 PSIG. The centralattachment arm 202 is adapted to be coupled to the robotic arm. In oneembodiment shown in FIG. 2, the robotic arm system 100 shown in FIG. 1is adapted to provide six degrees of freedom. These degrees of freedomare shown in FIG. 2 and include motion in the XYZ axis as well as roll(rotation about the Z-axis), pitch (rotation about the X-axis), and yaw(rotation about the Y-axis). The central attachment arm 202 is furthercoupled to the plate 206 such that motion of the central attachment arm202 correspondingly moves the plate 206 and the one or more panelingpickup tools 204 _(i), mounted thereon. The central attachment arm 202is further coupled to a first actuator 208 which is then coupled to afirst arm 210. The first actuator 208 is a rotational actuator, e.g., astepper motor, known to a person having ordinary skill in the art, whichcan rotate the first arm 210 independent of the plate 206. The first arm210 is further coupled to a second actuator 212 which is anotherrotational actuator, e.g., a stepper motor. The second actuator 212 isfurther coupled to a second arm 214 which together with the first arm210, and the first actuator 208 provide two additional degrees offreedom for the panel handling and fastening system 150. Coupled to thesecond arm 214 are two additional actuators: a third actuator 216 whichselectively engages a tool 220 _(i). Example tools include a fasteningtool 220 ₁ (e.g., a rotatory driver), and a cutting tool 220 ₂ (e.g., acutting bit with an optional vacuum to first cut and further drawcutting dust from the surrounding air). The tools 220 _(i) are coupledto a rectilinear actuator 218 which allow a rectilinear advancement ofthe tools 220 _(i) utilizing, e.g., a conventional rack-and-pinionmechanism.

Referring to FIGS. 3, 4, and 5 operation of the robotic arm system 100is provided. Referring to FIG. 3, the robotic arm 120 positions thepanel handling and fastening system 150 above a stack of panels 302until the plate 206 (see FIG. 2) and the paneling pickup tools 204 _(i)are firmly and centrally positioned atop the panel 302. Once vacuum isapplied to the paneling pickup tools 204 _(i), the robotic arm 120positions the panel handling and fastening system 150 in the appropriateorientation and position (for example vertically for installation onto awall or horizontally for installation onto a ceiling), as shown in FIG.4, which shows the panel handling and fastening system 150 holding thepanel 302 vertically ready for attachment on to a framed wall. Referringto FIG. 5, the panel handling and fastening system 150 is shown placingthe panel 302 onto a wall above a second panel 302 which has alreadybeen installed. The framed wall includes a bottom plate 502 and a topplate 506, in between the two are a plurality of studs 504 which areplaced at a nominal and repeated distance from each other (e.g., 16″ onthe center). The already installed panel 302 is fastened with aplurality of fasteners 508 positioned according to a fastening schedulebased on an operation plan, discussed further below. FIG. 5, furthershows the panel handling and fastening system 150 in operation in a workarea paneling a framed wall 500 where the first actuator 208, and thesecond actuator 212 have been actuated such that the first arm 210, thesecond arm 214, and the third actuator are extended away from thecentral attachment arm 202 to the edge of the panel 302, placing a newfastener 508 _(n).

In this setting, the panel handling and fastening system 150 has appliedvacuum to the paneling pickup tools 204 _(i) (see FIG. 2) so as toenable the panel handling and fastening system 150 pick up the panel 302(see FIG. 4), utilize a vision system, described below, properly placethe panel 302 (see FIG. 4) against the framed wall 500 and beginfastening the panel 302 (see FIG. 4) against the framed wall 500.

The panel handling and fastening system 150 is adapted to cooperativelyinteract with a vision system 600. Referring to FIG. 6 a generalschematic of the vision system 600 is shown with respect to the panelhandling and fastening system 150. The vision system 600 may constitutetwo single-vision cameras 602 and 604. These single-vision cameras 602and 604 may be positioned at a predetermined distance away from theframed wall 500 and from the panel handling and fastening system 150.Using a coordinate translation approach, images from each of thesingle-vision cameras 602 and 604 can be used to establish a depth forobjects away from the panel handling and fastening system 150, as knownto a person having ordinary skill in the art. To utilize this coordinatetranslation approach, same objects in images from each of twosingle-vision cameras 602 and 604 are associated with one-another andthe depth of the object from a central point (e.g., a point on the panelhandling and fastening system 150) is thus calculated. According to oneembodiment, at least one of the two single-vision cameras 602 and 604 isan RGB (red, green, blue) camera. The purpose for this embodiment is toanalyze the color of the image from this RGB camera in order todetermine flaws in the construction material. It is common for woodenstuds 504 (see FIG. 5) to have knots. Alternatively, where there areplumbing fixtures and/or electrical conduits passing through studs 504(see FIG. 5), there may be metallic plates (not shown) pre-attached tothe studs (see FIG. 5) which represent locations where fasteners 508_(n) (see FIG. 5) should not be attached. To identify these features(i.e., fastener forbidden zones), a color analysis of the images of atleast one RGB camera is needed. To do this color analysis, each pixel ofeach image of the RGB camera is compared to neighboring pixels todetermine a difference greater than a predetermined threshold. Such adifference would represent a variance in material that can becategorized as defects, metallic plates, or other issues with the studs504 (see FIG. 5). The panel handling and fastening system 150 includes aprocessing unit, discussed below, which receives images from thesingle-vision cameras 602 and 604 and analyzes those images to determinesuch forbidden zones for fastening the panel 302 (see FIG. 5) to thestuds 504 (see FIG. 5). When such forbidden zones are determined by theprocessing unit, described below, the processing unit using the imagesfrom the single-vision cameras 602 and 604 determines the closestlocation on the studs 504 (see FIG. 5) which away from the establishedforbidden zone. This location may be above or below the pre-scheduledlocation for fastening the fastener 508 _(n) (see FIG. 5).

This mapping of the framed wall 500 is performed and logged into memoryof the processing unit, described below, prior to the placement of thepanel 302 (see FIG. 5) onto the framed wall 500. This sequence ofimaging and logging forbidden zones according to the present disclosure,is extremely advantageous as a manual operation for hanging panels thatrepresents the state of the art for the last several decades suffersfrom not knowing the location of these defects and/or electrical andplumbing fixtures, once the panel is placed on the framed wall. Thischallenge with the existing prior art is because once the panel isplaced on the framed wall, the panel blinds these areas from sight ofthe workers. Therefore, the workers have to make exact measurements asto where such forbidden zones are. For example on a 4 foot by 10 footpanel, the worker has to make a precise measurement on a stud that isonly 1½″ wide, representing an exceedingly difficult task.

Referring back to FIG. 5, this operation of avoiding a forbidden zone isdepicted. Suppose the panel handling and fastening system 150 isscheduled to place a fastener 508 _(i) at a predetermined location onthe stud 508 _(i). Prior to attaching the panel 302 _(i) to the famedwall 500, the vision system 600 (see FIG. 6) generates an image of thestud 508 _(i) which is analyzed by the processing system, describedbelow, as having a deformity 510, e.g. a hole or a knot, in the stud 504_(i) which is a common occurrence with wooden studs. The location of thedeformity happens to coincide with the predetermined location offastening the fastener 508 _(i). The processing system, described below,based on proximity to other deformities or other obstructions, such asan electrical or plumbing fixture may determine to place the fastener508 _(i) at a higher position and further provide an additional fastener508 _(i+1) at a lower position as compared to deformity 510.

Referring back to FIG. 6, the vision system 600 also logs into memory ofthe processing unit, described below, the location of electricaljunction boxes (in construction terminology referred to as gangboxes). Agangbox is often installed on a stud with only a fraction of the gangboxprotruding away from the stud. This type of disposition, represents amore serious challenge for panel hangers of the prior art. The challengestems from not knowing the location of the gangbox and thusinadvertently paneling over the gangbox. Thereafter, when an electricianreturns to complete the final electrical connections, the gangbox ishidden behind the panel. This requires the electrician to search for thegangbox by making exploratory holes in the panel which will need to befilled and patched.

The vision system 600 of the present disclosure alleviates theaforementioned challenge. Each gangbox is identified via an imageanalysis of the single-vision cameras 602 and 604 and the outline ofeach gangbox is identified on the studs. This identification is shown inFIG. 5. A gangbox 510 is shown on a stud 504. Prior to attachment of thepanel 302 onto the framed wall 500, an outline 512 of the gangbox 510 isanalyzed from the single-vision cameras 602 and 604 and logged intomemory of the processing unit, described below, for later processing(including cutting around the gangbox 510 about its outline 512, asdescribed further below).

While two single-vision cameras 602 and 604 are shown in FIG. 6, analternative approach may also be implemented with a single RGB-Depthcamera 606. An RGB-Depth camera, not only generates a RGB image, but foreach pixel generates a distance away from the camera of the object ofwhich the pixel represents. That is, instead of only a triple datasetoutput for each pixel (one for red, one for green, and one for blue),generally obtained from a regular RGB camera (e.g., one or both of thesingle-vision cameras 602 and 604 as discussed above according to oneembodiment), the RGB-Depth camera 606 provides a dataset including fourpieces of information including RGB and depth of the pixel. ThisRGB-Depth camera 606 can be installed on the panel handling andfastening system 150, as shown in FIG. 6, or at a fixed location away(not shown) from the panel handling and fastening system 150. In eithercase, using a coordinate translation technique, known to a person havingordinary skill in the art, the depth of each stud 504 (see FIG. 5), andthe characteristics of each stud 504 (see FIG. 5) including locations ofknots, holes, electrical and plumbing fixtures, and gangboxes can beascertained by the processing unit, described below, and logged intomemory prior to attaching a corresponding panel 302 (see FIG. 5) ontothe framed wall 500.

As described above, and further with reference back to FIG. 2, tools 220_(i) are used to attach and augment the panel 302 (see FIG. 5). One suchtool 220 ₂ is a cutting tool which includes a specialty drill bit thatis used for cutting panels 302 (see FIG. 5). From time-to-time, a panel302 (see FIG. 5) may need to be augmented. This augmentation may includecutting the size of the panel 302 (see FIG. 5) to fit an area smallerthan the un-augmented size of the panel 302 (see FIG. 5); or to reveal agangbox 510 (see FIG. 5). For revealing a gangbox 510 (see FIG. 5), thespecialty drill bit of the tool 220 ₂ includes a non-cutting end that isintended to pierce the panel 302 (see FIG. 5) and while in contact withoutside of the outline 512 (see FIG. 5) of the gangbox 510 (see FIG. 5)trace the outline 512 (see FIG. 5) while cutting the panel 302 (see FIG.5). The tool 202 _(i) may further include a vacuum attachment so as toremove cutting dust from the surrounding air while cutting, as describedbelow. In addition, the tool 220 _(i) may include a pickup tool (notshown) similar to the pickup tool 204 _(i) for capturing the cut portionand disposing of that cut portion in a disposal area.

The tool 202 _(i) may also be used to cut the panel 302 (see FIG. 5)down to an appropriate size. This cutting can be performed by placingoversized panel 302 (see FIG. 5) on the framed wall 500 using only a fewfasteners 508 (see FIG. 5), in a manner that allows the oversized panelto overhang previously fastened panels. Then the panel handling andfastening system 150 can determine the precise position of cuts by theprocessing unit, described below, using the tool 220 ₂ cut the excesspart of the panel, optionally vacuuming dust from the surrounding air,and remove the cut portion into the disposal area. The panel handlingand fastening system 150 can then proceed by fastening the additionalfasteners 508 (see FIG. 5) according to the previously determinedfastening pattern determined by the processing unit, described below.

Referring to FIG. 7, a block diagram of the data processing system 700is shown. The data processing system 700 includes a processing unit 702that includes memory and other associated blocks that is further definedwith reference to FIG. 8, described below. The processing unit 702, asdescribed above, is responsible for a variety of different processingtasks. As an initial matter, the processing unit via an input/outputinterface, further described in reference to FIG. 8, receives a generalplan of panel installation on inside and outside walls of a constructionproject. This input is identified as 704 in FIG. 7. In addition, asdescribed above, the vision system 600 provides images including allblack/white, all color, one black/white and one color, one black/whitewith depth, or one color with depth to the processing unit 702. Asdescribed above, the processing unit 702 analyzes these images anddetermines i) position of the framed wall 500 (see FIG. 5), the positionof each stud 504 (see FIG. 5), the position of each plate 502 and 506(see FIG. 5), identify and determine position of any deformity 510 (seeFIG. 5), position of any electrical and plumbing fixture (not shown),and position of each gangbox 510 (see FIG. 5) and the associated outline512 (see FIG. 5). Once these features, positions, and dimensions havebeen determined by the processing unit 702, a plan is generated on howto proceed with fastening the panels 302 (see FIG. 5) to the framed wall500 (see FIG. 5).

With continued reference to FIG. 7, an optional building informationmodel (BIM) dataset 706 is also shown which provides input to theprocessing unit 702. A BIM dataset is known to a person having ordinaryskill in the art. A BIM dataset includes detailed information aboutpositions of framed walls, including positions of the associated studs,gangboxes, electrical, mechanical, and plumbing fixtures, as well as avariety of other information. The processing unit 702 is adapted toreceive the BIM dataset 706 and use that in connection with the visionsystem 600 as well as feedback sensors (including stud finder and aforces sensor (not shown) as one of the tools 220 _(i) (see FIG. 2)) toestablish a report of quality of building based on variances between theBIM dataset 706 and the signals received and interpreted from theaforementioned feedback sensors 220 _(i) (see FIG. 2) and the visionsystem 600 representing actual positions of the construction material.

It should be appreciated that in order to match data from the BIMdataset 706 and the position of building material determined by theprocessing block 702, an association between respective coordinatesystems must be created. For example, the BIM dataset 706 may be basedon a global coordinate system as well as local coordinate systems. Forexample, the global coordinate system may include an origin at a corner(not shown) of a room (not shown), and a local coordinate system foreach framed wall (e.g., the framed wall 500 (see FIG. 5)) with an origindefined at a lower left corner of each such framed wall. Such coordinatesystems must be translated to the coordinate system of the processingunit 702, according to coordinate translation methods known to a personhaving ordinary skill in the art, prior to determining variations thatinform the quality report.

With the fastening plan generated, the processing unit 702 provides datato drivers (not shown) of actuators (e.g., 208, 212, 216, and 218) andreceive signals from the feedback sensors, as discussed above, as showncollectively in block 710. Furthermore, the processing unit 702 providesmotion requests to the robotic arm 120 and receives communication backfrom the robotic arm 120. According to one embodiment, the processingunit 702 may be adapted to provide detailed instructions to the roboticarm 120 including low-level actuator information and thus control therobotic arm 120 at a low-level. In this embodiment, the robotic arm maynot have a processing block of its own and rely on the processing unit702 to accomplish all of the necessary calculations of all actuators ofthe robotic arm 120. In another embodiment, the processing unit 702cooperates with a separate processing block (not shown) of the roboticarm 120 such that the processing unit 702 provides desired coordinatesfor the end point of the central attachment arm 202 (see FIG. 2), andrely on the processing block (not shown) of the robotic arm 120 tomanage the position calculations.

Referring to FIG. 8, an example of a computer system is provided thatcan interface with the above-discussed data processing system 700.Referring to FIG. 8, a high-level diagram is provided showing thecomponents of an exemplary data-processing system 1000 for analyzingdata and performing other analyses described herein, and relatedcomponents. The system includes a processor 1086 which is part of theprocessing unit 702 (see FIG. 7), a peripheral system 1020, a userinterface system 1030, and a data storage system 1040. The peripheralsystem 1020, the user interface system 1030 and the data storage system1040 are communicatively connected to the processor 1086. Processor 1086can be communicatively connected to network 1050 (shown in phantom),e.g., the Internet or a leased line, as discussed below. The imagingdescribed in the present disclosure may be obtained using imagingsensors 1021 and/or displayed using display units (included in userinterface system 1030) which can each include one or more of systems1086, 1020, 1030, 1040, and can each connect to one or more network(s)1050. Processor 1086, and other processing devices described herein, caneach include one or more microprocessors, microcontrollers,field-programmable gate arrays (FPGAs), application-specific integratedcircuits (ASICs), programmable logic devices (PLDs), programmable logicarrays (PLAs), programmable array logic devices (PALs), or digitalsignal processors (DSPs).

Processor 1086 can implement processes of various aspects describedherein. Processor 1086 can be or include one or more device(s) forautomatically operating on data, e.g., a central processing unit (CPU),microcontroller (MCU), desktop computer, laptop computer, mainframecomputer, personal digital assistant, digital camera, cellular phone,smartphone, or any other device for processing data, managing data, orhandling data, whether implemented with electrical, magnetic, optical,biological components, or otherwise. Processor 1086 can includeHarvard-architecture components, modified-Harvard-architecturecomponents, or Von-Neumann-architecture components.

The phrase “communicatively connected” includes any type of connection,wired or wireless, for communicating data between devices or processors.These devices or processors can be located in physical proximity or not.For example, subsystems such as peripheral system 1020, user interfacesystem 1030, and data storage system 1040 are shown separately from thedata processing system 1086 but can be stored completely or partiallywithin the data processing system 1086.

The peripheral system 1020 can include one or more devices configured toprovide digital content records to the processor 1086. For example, theperipheral system 1020 can include digital still cameras, digital videocameras, cellular phones, or other data processors. The processor 1086,upon receipt of digital content records from a device in the peripheralsystem 1020, can store such digital content records in the data storagesystem 1040.

The user interface system 1030 can include a mouse, a keyboard, anothercomputer (connected, e.g., via a network or a null-modem cable), or anydevice or combination of devices from which data is input to theprocessor 1086. The user interface system 1030 also can include adisplay device, a processor-accessible memory, or any device orcombination of devices to which data is output by the processor 1086.The user interface system 1030 and the data storage system 1040 canshare a processor-accessible memory.

In various aspects, processor 1086 includes or is connected tocommunication interface 1015 that is coupled via network link 1016(shown in phantom) to network 1050. For example, communication interface1015 can include an integrated services digital network (ISDN) terminaladapter or a modem to communicate data via a telephone line; a networkinterface to communicate data via a local-area network (LAN), e.g., anEthernet LAN, or wide-area network (WAN); or a radio to communicate datavia a wireless link, e.g., WiFi or GSM. Communication interface 1015sends and receives electrical, electromagnetic or optical signals thatcarry digital or analog data streams representing various types ofinformation across network link 1016 to network 1050. Network link 1016can be connected to network 1050 via a switch, gateway, hub, router, orother networking device.

Processor 1086 can send messages and receive data, including programcode, through network 1050, network link 1016 and communicationinterface 1015. For example, a server can store requested code for anapplication program (e.g., a JAVA applet) on a tangible non-volatilecomputer-readable storage medium to which it is connected. The servercan retrieve the code from the medium and transmit it through network1050 to communication interface 1015. The received code can be executedby processor 1086 as it is received, or stored in data storage system1040 for later execution.

Data storage system 1040 can include or be communicatively connectedwith one or more processor-accessible memories configured to storeinformation. The memories can be, e.g., within a chassis or as parts ofa distributed system. The phrase “processor-accessible memory” isintended to include any data storage device to or from which processor1086 can transfer data (using appropriate components of peripheralsystem 1020), whether volatile or nonvolatile; removable or fixed;electronic, magnetic, optical, chemical, mechanical, or otherwise.Exemplary processor-accessible memories include but are not limited to:registers, floppy disks, hard disks, tapes, bar codes, Compact Discs,DVDs, read-only memories (ROM), erasable programmable read-only memories(EPROM, EEPROM, or Flash), and random-access memories (RAMs). One of theprocessor-accessible memories in the data storage system 1040 can be atangible non-transitory computer-readable storage medium, i.e., anon-transitory device or article of manufacture that participates instoring instructions that can be provided to processor 1086 forexecution.

In an example, data storage system 1040 includes code memory 1041, e.g.,a RAM, and disk 1043, e.g., a tangible computer-readable rotationalstorage device such as a hard drive. Computer program instructions areread into code memory 1041 from disk 1043. Processor 1086 then executesone or more sequences of the computer program instructions loaded intocode memory 1041, as a result performing process steps described herein.In this way, processor 1086 carries out a computer implemented process.For example, steps of methods described herein, blocks of the flowchartillustrations or block diagrams herein, and combinations of those, canbe implemented by computer program instructions. Code memory 1041 canalso store data, or can store only code.

Various aspects described herein may be embodied as systems or methods.Accordingly, various aspects herein may take the form of an entirelyhardware aspect, an entirely software aspect (including firmware,resident software, micro-code, etc.), or an aspect combining softwareand hardware aspects. These aspects can all generally be referred toherein as a “service,” “circuit,” “circuitry,” “module,” or “system.”

Furthermore, various aspects herein may be embodied as computer programproducts including computer readable program code stored on a tangiblenon-transitory computer readable medium. Such a medium can bemanufactured as is conventional for such articles, e.g., by pressing aCD-ROM. The program code includes computer program instructions that canbe loaded into processor 1086 (and possibly also other processors), tocause functions, acts, or operational steps of various aspects herein tobe performed by the processor 1086 (or other processors). Computerprogram code for carrying out operations for various aspects describedherein may be written in any combination of one or more programminglanguage(s), and can be loaded from disk 1043 into code memory 1041 forexecution. The program code may execute, e.g., entirely on processor1086, partly on processor 1086 and partly on a remote computer connectedto network 1050, or entirely on the remote computer.

Those having ordinary skill in the art will recognize that numerousmodifications can be made to the specific implementations describedabove. The implementations should not be limited to the particularlimitations described. Other implementations may be possible.

1. An autonomous construction robotic system, comprising: a processing unit; a robotic arm, the robotic arm adapted to be coupled to a central attachment arm and thereby position the central attachment arm according to a plurality of degrees of freedom; a panel handling and fastening system, including a panel handling assembly coupled to the central attachment arm and adapted to pick and place a construction panel onto a framed structure within a construction zone; and a vision system adapted to provide visual information to the processing unit associated with the framed structure, wherein the processing unit processes the visual information to automatically determine placement position of the construction panel on the framed structure.
 2. The autonomous construction robotic system of claim 1, the panel handling and fastening system further includes an end effector assembly coupled to the panel handling assembly and adapted to automatically fasten the construction panel to the framed structure according to a predetermined fastening schedule.
 3. The autonomous construction robotic system of claim 2, wherein the end effector assembly is coupled to the panel handling assembly via a first rotational actuator, a first arm, a second rotational actuator, and a second arm, wherein the first rotational actuator is adapted to rotate the first arm, the second rotational actuator, the second arm, and the end effector assembly independent of the panel handling assembly.
 4. The autonomous construction robotic system of claim 3, the end effector assembly includes a third rotational actuator adapted to present one or more tools for working the construction panel.
 5. The autonomous construction robotic system of claim 4, the end effector assembly further includes a rectilinear actuator coupled to the third rotational actuator and adapted to rectilinearly advance the one or more tools once the third rotational actuator has presented the one or more tools.
 6. The autonomous construction robotic system of claim 4, wherein the one or more tools includes a fastening tool adapted to fasten the construction panel to the framed structure.
 7. The autonomous construction robotic system of claim 6, wherein the one or more tools further includes a cutting tool adapted to cut a portion of the construction panel while the construction panel is at least partially fastened to the framed structure.
 8. The autonomous construction robotic system of claim 7, wherein the one or more tools further includes a stud finder to determine presence of studs once the construction panel is fastened to the framed structure.
 9. The autonomous construction robotic system of claim 7, wherein the one or more tools further includes a force sensor adapted to be positioned on a plurality of points on the framed structure, and wherein signal from the force sensor is processed by the processing unit to determine precise position of the framed structure.
 10. The autonomous construction robotic system of claim 1, the vision system includes at least one camera.
 11. The autonomous construction robotic system of claim 10, the at least one camera is a red-green-blue (RGB)-Depth camera, adapted to generate RGB images along with depth information of objects in the RGB images.
 12. The autonomous construction robotic system of claim 11, wherein the RGB images are analyzed by the processing unit to determine forbidden zones in the framed structure based on presence of one or more of deformities, presence of electrical fixtures, presence of mechanical fixtures, and presence of plumbing fixtures.
 13. The autonomous construction robotic system of claim 10, the at least one camera is at least two single-vision cameras adapted to provide images which when analyzed by the processing unit produce depth of objects in the images from a predetermined reference point.
 14. The autonomous construction robotic system of claim 13, wherein at least one of the at least two single-vision cameras is an RGB camera adapted to generate RGB images.
 15. The autonomous construction robotic system of claim 14, wherein the RGB images are analyzed by the processing unit to determine forbidden zones in the framed structure based on presence of one or more of deformities, presence of electrical fixtures, presence of mechanical fixtures, and presence of plumbing fixtures.
 16. The autonomous construction robotic system of claim 1, wherein the processing unit is adapted to receive a building information modeling (BIM) dataset including data associated with the framed structure, wherein the processing unit is adapted to compare data from the BIM to the processed data received from the vision system.
 17. The autonomous construction robotic system of claim 1, wherein the processing unit is further adapted to generate a report based on said comparison.
 18. The autonomous construction robotic system of claim 9, wherein the processing unit is adapted to receive a building information modeling (BIM) dataset including data associated with the framed structure, wherein the processing unit is adapted to compare data from the BIM to the processed signal received from the force sensor.
 19. The autonomous construction robotic system of claim 18, wherein the processing unit is further adapted to generate a report based on said comparison.
 20. The autonomous construction robotic system of claim 1, wherein the panel handling assembly includes a plate and a plurality of paneling pickup tools including one or more suction cups coupled to a vacuum pump. 